High Efficiency Video Coding (HEVC) is a new video coding standard currently being developed in Joint Collaborative Team-Video Coding (JCT-VC). JCT-VC is a collaborative project between Moving Picture Experts Group (MPEG) and International Telecommunication Union Telecommunication standardization sector (ITU-T). Currently, a Committee Draft (CD) is defined that includes a number of new tools and is considerably more efficient than H.264 Advanced Video Coding (AVC).
It is likely that extensions of HEVC will be defined in the future, such as a multi-view extension and a scalable extension. These extensions could then contain data relating to, for instance, additional views depending on a base view or additional layers depending on a base layer.
It can be assumed that backwards compatibility for some parts of an encoded sequence, e.g. the base view or the base layer, is a requirement. However, it is important that a legacy, i.e. base specification compliant, decoder is able to correctly handle the bitstream even when using such extensions.
HEVC defines some Network Abstraction Layer (NAL) unit types to be reserved for, among others, extensions. Also parameter sets, e.g. Sequence Parameter Set (SPS), Picture Parameter Set (PPS) and Adaptation Parameter Set (APS), contain extension fields at the end of their syntax tables. It is specified that a decoder conforming to the base specification of HEVC shall ignore the data in the extension field at the end of a parameter set. The syntax for the extension data is shown below in an illustrative example in the form of SPS as parameter set. The syntax would be similar if using other parameter sets than SPS. The total length of a parameter set is known by the decoder through the syntax element NumBytesInRBSP, typically provided by the system layer, so the decoder will have no problems in detecting where the extension field ends.
seq_parameter_set_rbsp( ) {Descriptor. . .sps_extension_flagu(1)if( sps_extension_flag )white( more_rbsp_data( ) )sps_extension_data_flagu(1)rbsp_trailing_bits( )}
A NAL unit that contains an encoded slice of a picture consists of two parts: first the slice header and then the slice data. There is currently no extension field for encoded slices.
The slice header contains information about the current slice, e.g. slice type, which reference pictures to be used, etc. There is a need for adding extensions and extension fields in HEVC. However, in such a case it is important that both a legacy decoder conforming to the base specification, i.e. not using extensions, and an extension-compliant decoder could correctly handle a bitstream comprising such extensions.
The current layout of a slice header for an encoded representation of a slice makes this need hard or even impossible to implement when having both legacy and extension-compliant decoders.